feat: update the benchmark dataset and generator to add runner to the data id#7498
feat: update the benchmark dataset and generator to add runner to the data id#7498joseph-isaacs wants to merge 5 commits intodevelopfrom
Conversation
|
this requires rebasing, the script doesn't exist after #7466 |
…r-more Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> # Conflicts: # .github/scripts/run-sql-bench.sh # .github/workflows/sql-benchmarks.yml # vortex-bench/Cargo.toml # vortex-bench/src/runner.rs
|
If benchmarks pass, lets merge it |
Polar Signals Profiling ResultsLatest Run
Previous Runs (1)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
|
File Sizes: PolarSignals ProfilingFile Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVMEVortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
datafusion / arrow (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
duckdb / duckdb (no group data, 0↑ 0↓)
|
File Sizes: TPC-H SF=1 on NVMEFile Size Changes (18 files changed, +0.0% overall, 18↑ 0↓)
Totals:
|
Benchmarks: FineWeb NVMeVortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
|
File Sizes: FineWeb NVMeFile Size Changes (2 files changed, +0.0% overall, 2↑ 0↓)
Totals:
|
|
@joseph-isaacs this breaks the baseline |
AdamGS
left a comment
There was a problem hiding this comment.
This shouldn't invalidate all the history, either migrate it somehow or handle it during processing
Benchmarks: TPC-DS SF=1 on NVMEVortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
duckdb / duckdb (no group data, 0↑ 0↓)
|
File Sizes: TPC-DS SF=1 on NVMEFile Size Changes (48 files changed, +0.0% overall, 48↑ 0↓)
Totals:
|
Benchmarks: FineWeb S3Vortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
|
Benchmarks: TPC-H SF=10 on NVMEVortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
datafusion / arrow (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
duckdb / duckdb (no group data, 0↑ 0↓)
|
File Sizes: TPC-H SF=10 on NVMEFile Size Changes (48 files changed, +0.0% overall, 48↑ 0↓)
Totals:
|
Benchmarks: Statistical and Population GeneticsVortex (geomean): no vortex data duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
|
File Sizes: Statistical and Population GeneticsFile Size Changes (2 files changed, +0.0% overall, 2↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3Vortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
|
🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨Benchmark |
Benchmarks: TPC-H SF=10 on S3Vortex (geomean): no vortex data datafusion / vortex-file-compressed (no group data, 0↑ 0↓)
datafusion / vortex-compact (no group data, 0↑ 0↓)
datafusion / parquet (no group data, 0↑ 0↓)
duckdb / vortex-file-compressed (no group data, 0↑ 0↓)
duckdb / vortex-compact (no group data, 0↑ 0↓)
duckdb / parquet (no group data, 0↑ 0↓)
|
|
I should have mentioned that I would migrate the data manually |
AdamGS
left a comment
There was a problem hiding this comment.
we can move the runner info into QueryMeasurement, just makes it easier to process later
|
superseded by #7622 |
e.g.
tpch/sf_1/q01/ec2_c6id.8xlarge/datafusion:vortex-file-compressedThere is a local migration script.